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About Research for Action 

Research for Action (RFA) is a Philadelphia-based nonprofit organization. We seek to use research as 
the basis for the improvement of educational opportunities and outcomes for traditionally underserved 
students. Our work is designed to strengthen public schools and postsecondaiy institutions; provide 
research-based recommendations to policymakers, practitioners and the public at the local, state and 
national levels; and enrich the civic and community dialogue about public education. For more 
information, please visit our website at www.researchforaction.org . 
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Introduction 

To support the implementation of the Common Core State Standards (CCSS), the Bill and Melinda 
Gates Foundation invested in the development and dissemination of two tools aimed at 
operationalizing classroom instruction based on the standards: the Literacy Design Collaborative 
(LDC)’s Framework and the Math Design Collaborative (MDC)’s Formative Assessment Lessons. These 
tools stress teachers’ attention to high quality instructional tasks (City, Elmore, Fiarman, & Teitel, 
2010; Hiebert & Carpenter, 1992; Hiebert & Wearne, 1993; Jones, Valdez, Nowakowski & Rasmussen, 
1994)> use of formative assessments embedded in those tasks (Black, Harrison, Lee, Marshall & 
William., 2004; Clarke & Shinn, 2004; Fuchs, 2004; Tunstall, 1996), and professional learning 
opportunities that focus on both content knowledge and instruction (Birman, Desimone, Porter & 
Garet, 2000; Cohen & Hill, 1997; Kennedy, 1998). 

Experts from the Literacy Design Collaborative (LDC) have developed a Framework that can be 
customized by English Language Arts (ELA), social studies and science teachers into writing tasks 
designed to facilitate CCSS-based student literacy and content learning. LDC also developed a module 
structure that teachers can use to create a plan for teaching students the content and literacy skills 
necessary to complete the writing task. Eor example, a social studies teacher might teach a template 
task that asks students to write an argumentative essay on how the political views of the signers of the 
Constitution impacted the American political system. The entire process takes approximately two to 
three weeks of classroom time to complete. 

Similarly, experts from the Shell Centre have developed a set of Formative Assessment Lessons 
(Lessons) for secondary mathematics teachers to facilitate CCSS-based student mathematics learning 
and provide teachers with feedback about student understanding and mastery. Unlike the LDC 
Framework, the Lessons are not customizable and typically take a few days to complete rather than few 
weeks. Another key difference between the modules and the Formative Assessment Lessons is that 
modules can be utilized to teach new content whereas Formative Assessment Lessons are meant for 
gauging student mastery and/or reinforcing already-taught material. For this reason, Lessons are 
intended to be taught about three quarters of the way into a unit. 

Research for Action (RFA) began examining the implementation of this initiative in its pilot year of 
2010-2011 (Year One), and has continued this research into the 2011-2012 school year (Year Two), 
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which has seen the expansion of the use of the tools. The results of our research at the end of the second 
year of the initiative are presented in four Research Briefs, which are related but are also designed to 
stand alone: 


LDC and MDC: Implementation and Scale-Up Status 
Brief One. LDC and MDC: Theory of Action and the Landscape of Implementation 
Brief Two. Robust Implementation of LDC: Teacher Perceptions of Tool Use and Outcomes 
Brief Three. Robust Implementation of MDC: Teacher Perceptions of Tool Use and Outcomes 
Brief Four. Conditions for Scale and Sustainability 


The first of four briefs on the LDC/MDC initiatives, Brief One presents key background information 
that will be referred to throughout the other three briefs. To describe the implementation and the 
scale-up of the LDC/MDC initiative, this brief includes: 

• the Theory of Action, which provides a comprehensive organizing framework for the 
presentation of our research results; 

• a national overview of the initiative, highlighting the extent of training on the LDC and MDC 
tools across the United States; 

• a detailed look at changes to the number of schools and teachers involved across Research for 
Action’s four LDC and four MDC study sites; 

• a description of our research methodology; and, 

• a short summary of Briefs 2, 3, and 4. 


Theory of Action 

The Theory of Action depicts the underlying assumptions and conditions on which the LDC and MDC 
initiatives are based. Drawing on the goals of the LDC/MDC initiatives, our own research, and existing 
research on similar initiatives, the Theory of Action is presented below in Figure 1. Each element of the 
Figure is then described briefly. 
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Figure 1. Theory of Action 
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Context 

Although individual teachers use the tools in a classroom setting, contextual factors influence their use. 
The Theory of Action identifies broad categories of such contextual factors, including students, schools, 
districts, networks, regions, states, support organizations, and curriculum. 


Indicators of Robust Implementation 

RFA developed the indicators of Robust Implementation after a thorough analysis of data collected 
through interviews with PD providers and tool developers and review of program documents. This 
analysis informed the identification of robust indicators, as well as the larger theory of action for the 
initiative. Circling the conditions in the center of the Theory of Action, these indicators represent the 
use of the tools in the classroom, and the changes expected in teacher practice and student behavior as 
the tools are implemented. Robust implementation is manifested in two main arenas: Teacher Beliefs 
and Knowledge, and Classroom Changes. Each is described below. 

Figure 2. Indicators of Robust Implementation: Teacher Beliefs and Knowledge 


Robust 

Implementation 

Indicator 


Definition 


Teachers believe 
in the underlying 
principles of tools 


LDC: Traditionally, elementary English or Language Arts classes are responsible for 
teaching literacy. As secondary content-area teachers begin to include literacy instruction 
in their courses by using LDC, it is important for them to believe that literacy instruction 
is a valid and worthwhile responsibility. 

MDC: The use of the Formative Assessment Lessons requires teachers to adjust their 
math instruction to teach in fundamentally different ways than most teachers traditionally 
teach math. New approaches to instruction include the constant facilitation and 
assessment of student learning as opposed to providing direct instruction, and allowing 
students to struggle to develop their own conceptual understandings of mathematics. 


Teachers exhibit ■ 
high levels of buy- 
in to the initiatives ' 


Teacher buy-in to instructional and curricular initiatives is central to the success of any 
new reform. Teachers need to believe that the initiative itself and its supporting structures 
will provide them with the tools to help their students achieve at higher levels. 




Teachers know how 
to use the tools 




LDC: Successful use of the LDC Framework requires a strong understanding of how to 
develop and use the Framework, including mini-tasks and the instructional ladder. 

MDC: Successful use of the Formative Assessment Lessons requires a strong 
understanding of how to place the Lessons into larger math units and discern and respond 
to students’ mathematical misconceptions. 
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Figure 3. Indicators of Robust Implementation: Classroom Changes 
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Once teachers know how to build and use the tools, they need to execute new pedagogical 
methods in ways that change instructional practice. 


Students exhibit 
engagement 
during tool use 


Students must be responsive to, and engaged by, the new instructional practices in order 
for the initiative to achieve its goal of improved student learning. 


Teachers perceive 
improvement in 
student learning 


Teachers need to perceive improvement in student learning as a result of tool use. 


Conditions for Scale Up and Sustainability 

The Conditions used as the basis for the Theory of Action are drawn from RFA’s first year research and 
the literature on successful scale-up and sustainability initiatives.^ The three overlapping Conditions in 
the Theory of Action represent the web of organizational, policy, and professional learning supports 
necessary for implementing, sustaining, and growing the use of the tools. These conditions were first 
described in RFA’s 2011 reports on tool implementation, and have been revised to include the broader 
array of leadership necessary for more intensive initiative scale-up. ^ The three Conditions discussed in 
this year’s reports include the following: 


1 See Bodilly, Glennan, Kerr, & Galegher (2004); Cobb, McClain, Lamberg, & Dean (2003); Coburn (2003); and Datnow, Hubbard & Mehan 
(2002). 

2 The September 2011 RFA reports entitled Establishing a Strong Foundation (LDC and MDC), included four conditions: 1) robust district, 
regional, school network leadership, 2) strong school leadership, 3) meaningful professional learning opportunities and 4) alignment with the 
CCSS, curricula and assessment. 
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Alignment: In order for the reform to be successful, it needs to be in alignment 
with other policies and initiatives taking place in the state, districts and schools 
where the reform is being implemented. If initiatives and policies are at cross- 
purposes, it becomes difficult to progress in any one direction. Because the LDC and 
MDC tools were designed to align with the Common Core State Standards (CCSS), 
alignment with these standards, as well as with other local curricula, and state and 
local assessments, is important to successful implementation and scale-up of the 
tools. 


Effective Leadership: 
• State/Regional 
• District/Network 
•Schoof 


Alignment with 

Meaningful, 

CCSS, local 

ongoing 

curricula 

professional 

and assessments 

B learning 


W opportunities 


Effective Leadership: Effective leaders at all levels, including the state, region, 
district/network and school, need to champion the initiative, provide needed 
resources and training, and help teachers understand how it fits into an overall plan 
for educational improvement. 



Professional Learning Opportunities: Teachers and leaders need meaningful 
and ongoing professional development and technical assistance to understand the 
purpose of the tools, how to implement them in the classroom, and refine their 
practice as they move forward. Along with formal professional development 
sessions, this construct also includes more informal work between colleagues in the 
school setting on a regular basis. 


Intermediate Outcomes 

When robust implementation of the tools occurs, we would expect to see two major, measurable 
outcomes emerge prior to the ultimate goal of graduating students who are college and career ready. 
These intermediate outcomes are: 

Broad and Deep Instructional Change. As described above, robust implementation includes 
changes in both teacher beliefs and knowledge, and changes in the classroom. As these changes take 
hold and deepen, teachers will exhibit significant changes in their pedagogy that will extend beyond the 
confines of the initiative, and into their general classroom practices. 

Increased Student Learning. Effective use of the instructional tools aligned to the CCSS will result 
in improvement in student learning that can be documented in a variety of ways, including traditional 
standardized student assessments, as well as reviews of student work by disinterested third-party 
researchers. 
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Long Term Impact: College and Career Readiness 

As students progress through schools that effectively utilize the LDC and MDC tools, their learning will 
become increasingly aligned with the Common Core State Standards. As a result, they will graduate 
from high school fully ready for college and career. 


National Perspective and a closer look at RFA's study sites 

During Year Two, RFA examined both the implementation of the tools and how their use had been 
scaled-up across sites. Indeed, the number of teachers and schools using the tools has expanded across 
the country .3 Figure 4 and 5 below display the states in which teachers have been trained in the LDC 
and MDC tools over the past two years, and provide an indication of the approximate number of 
teachers receiving professional development in each state .4 

Figure 4. LDC Tools Figure 5. MDC Tools 



1 1 no teachers trained 

1 1 no teachers trained 

1 1 1-20 teachers trained 

1 1 1-20 teachers trained 

1 1 21-100 teachers trained 

y 21-100 teachers trained 

1 over 100 teachers trained 

over 100 teachers trained 


During the 2011-2012 school year, RFA examined a subset of these states and specific districts and 
networks implementing the tools to understand how implementation and scale-up varied across 
different types of settings (e.g., location, size, implementation and scale-up patterns). In consultation 
with the Gates Foundation, the tool developers and the professional development providers, we 
carefully chose four LDC and four MDC study sites to conduct both the quantitative and qualitative 
research in Year Two. We decided to continue the research in six Year One study sites in order to 
examine implementation and scale-up over time, and included two new study sites in Year Two - one 
for LDC and one for MDC (New Visions and Jefferson County, respectively). 


3 LDC and MDC trainings vary across states, districts and schools. However, this data presents a national picture of the scope of training in the 
initiatives. 

4 States that have opted out of the initiative in Year Two, but have been trained in Year One are not included in the map. 
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The four figures below depict the change in number of teachers and schools involved from Year One and 
Year Two across LDC and MDC study sites. In the following figures, the key “2010-2011” includes all 
teachers and schools involved in Year One of the LDC/MDC initiative, and the key “2011-2012” includes 
all (both new and experienced) teachers and schools involved in the initiative in Year Two. 


Figure 6. LDC teacher scale-up in RFA research sites 


Figure 7. MDC teacher scale-up in RFA research sites 
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Figure 8. LDC school scale-up in RFA research sites Figure 9. MDC school scale-up in RFA research sites 



Overall, the number of teachers and schools involved in the LDC and MDC initiative increased from 
Year One to Year Two across most of the RFA study sites. However, it is important to note that each site 
scaled-up the use of the tool in different ways; some sites added more schools and/or districts, whereas 
others increased the number of teachers involved in the initiative while keeping the number of schools 
constant. The increase in number of LDC teachers across RFA’s study sites is particularly noteworthy. 
Increases in the number of schools and teachers involved in MDC in Year Two study sites was not as 
robust for a number of reasons: in Kenton County, the MDC initiative was already scaled-up during the 
2010-11 school year, and one of the other MDC study sites (Jefferson County) was in its first year of 
implementation. 
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Research Methodology 

RFA used both quantitative and qualitative research methods to inform our analysis: 

• Quantitative research activities include administering and analyzing four different surveys 
of LDC teachers, MDC teachers, principals, and district leaders. 

• Qualitative research activities include interviewing teachers, principals, district and state 
leaders, professional development providers and tool developers involved in LDC and/or MDC, 
as well as classroom and professional development observations. 

Surveys were administered to teachers and school and district administrators to depict a 
comprehensive picture of the LDC/MDC implementation landscape in RFA’s Year Two study sites. All 
459 LDC teachers involved in our four LDC study sites (17% of all LDC teachers trained in 2011-12 
school year) and all 180 MDC teachers involved in our four MDC study sites (12% of all MDC teachers 
trained in 2011-12 school year) were administered a survey. Additionally, 114 principals and 122 district 
leaders received a survey that focused on leadership, capacity, professional development, and scale. 
Response rates for the LDC and MDC teacher surveys were 53% and 54% respectively. Response rates 
for the school and district administrator surveys were 57% and 71%, respectively.. (Details on survey 
methodology are provided in the Technical Appendix A.) 

Further, a subsample of 120 teachers, 26 school administrators, and 35 district/state leaders were 
interviewed either in person or via phone. Figure 10 summarizes RFA’s Research Activities from both 
years. 


Figure 10. Research activities during Year One (2010-2011) and Year Two (2012-2012) 


Level of Data Collection 

Research Conducted; 
Year One 

Research Conducted; 
Year Two 

Tool developers (LDC and MDC) 

5 interviews 

6 interviews 

Professional development providers 

2 interviews 

4 interviews 

Professional development observations 

15 observations 

9 observations 

State-level policymakers & partners 

5 interviews 

15 interviews 

District/regional & network leadership 

15 interviews 

20 interviews 

School administration 

29 interviews 

26 interviews 

Teachers 

121 Interviews and 
37 observations 

120 Interviews and 
65 observations 
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Surveys 

Teachers: 179 

Teachers: 336 

2010-2011:(9 Districts) 

(Response rates: 

(Response rates: 

2011-2012: (39 Districts) 

LDC: 71%; MDC 53%) 

LDC: 53%; MDC: 54%) 
Principals: 65 
(Response rate: 57%) 
District Admin: 75 
(Response rate: 71%) 


Whaf s Next? 

The remaining three briefs take a closer look at the LDC and MDC study sites to identify the conditions 
and contexts in which the use of the tools can be continuously sustained and scaled up in the future. 
Specifically: 

• Brief Two: Robust Implementation of LDC: Teaeher Pereeptions of Tool Use and 
Outcomes discusses the ways in which the LDC framework has been implemented during Year 
Two and examines the indicators of robust implementation related to teacher beliefs and 
knowledge and classroom changes. 

• Brief Three: Robust Implementation of MDC: Teacber Perceptions of Tool Use and 
Outcomes discusses the ways in which the MDC initiative has been implemented during Year 
Two and examines the indicators of robust implementation related to teacher beliefs and 
knowledge and classroom changes. 

• Brief Four: Conditions for Scale and Sustainability defines scale-up and sustainability, 
discusses the conditions needed for successful scale-up of the tools, and highlights the lessons 
learned from the research on scale-up strategies to provide suggestions for the implementation 
of the tools in new and continuing sites. 

We look forward to continuing our analysis of the implementation and impacts of the LDC and 
MDC initiatives. Future work will focus on scale-up and sustainability, with the aim of providing key 
stakeholders with the information necessary to make decisions that best support educators in 
guiding students in their preparation for college and careers beyond high school. To address the 
heightened focus on scale-up and sustainability, our upcoming research activities will expand upon 
activities from this year (i.e., surveys, interviews, and observations) by reaching out to a greater 
number of participants. Also, we will conduct case studies to provide detailed descriptions of tool 
implementation and scale -up in four study sites. Finally, RFA will continue to work with CRESST 
to support their investigation of the relationship between the LDC and MDC initiatives and student 
outcomes and to develop systems to be used by teachers and administrators in observing and 
evaluating LDC and MDC tool implementation. 
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